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Abstract 

A simple geometrical characterization of configuration space neighborhoods 
of local energy minima in spin glass landscapes is found by exhaustive 
search. Combined with previous Monte Carlo investigations of thermal do- 
main growth, it allows a discussion of the connection between real and con- 
figuration space descriptions of low temperature relaxational dynamics. We 
argue that the part of state-space corresponding to a single growing domain is 
adequately modeled by a hierarchically organized set of states and that ther- 
mal (meta)stability in spin glasses is related to the nearly exponential local 

density of states present within each trap. 
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I. INTRODUCTION 



In this paper we consider the relation between configuration space geometry, excitation 
morphology and relaxation dynamics of spin glass systems. Spin glasses have a complex 
landscape fraught with local (free) energy minima, which, at sufficiently low temperatures, 
prevent the system from reaching a state of global thermal equilibrium: slow relaxation 
becomes the dominating observable features of the system on relevant time scales, and thus 
a main point of theoretical interest. Despite the absence of global equilibration, spin glasses 
- as well as other complex systems displaying broken ergodicity relax through a series 
of quasiequilibria involving subsets of configuration space (traps or metastable regions). 
Hence, during relevant experimental time windows measurable quantities appear as canonical 
averages restricted to the configurations of the trap, and the latter can be considered as a 
single point in the free energy landscape of the system. Free energy barriers separating 
metastable regions can be probed by so called aging experiments In Zero Field Cooled 
(ZFC) experiments ||, the system is rapidly cooled in zero field to a temperature T m and is 
then left undisturbed for a time t w . Thereafter, a small magnetic field is turned on, and the 
growing magnetic response of the system is measured. In Thermo Remanent Magnetization 
(TRM) experiments || the system is quickly cooled to T m and allowed to relax in a magnetic 
field for a time t w . The field is then cut and the decay of the magnetization is measured. 
In both cases the response curve strongly depends on the time t w elapsed previous to the 
field change. Qualitatively, this can be understood by the idea of partial equilibration 
just explained: after relaxing for a time t w the system occupies a valley in state space 
with a characteristic size related to t w . Probing the system on times scales less than t w 
reveals properties of the equilibrium fluctuations in this valley, while probing on longer time 
scales gives the non-equilibrium properties ||. Numerically, a coarse grained picture of 
configuration space relaxation can be obtained directly for microscopic systems of modest 
size |7],|[ by constructing the connectivity matrix of a subset of state space and solving the 
master equation for the probability flow. For each time, configurations mutually close to 
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equilibrium are lumped together, yielding a relaxation tree which describes the merging in 
time of larger and larger subset of internally equilibrated states. 

Theories of spin glass dynamics are either based on assumed scaling properties of low 
energy excitations in real space p|-|TT||, or on mesoscopic hierarchical models of state space, 
reviewed for instance in Ref. ||12||. Some of the approaches in the latter category are inspired 



by the equilibrium properties of mean field models fl3fl , whose relevance for short-range 



spin glasses is now hotly debated [14]]. However, hierarchical models can also simply be 



viewed |l5j as paradigms of sequential, strongly constrained dynamics, without reference to 
any equilibrium property. The models generally feature a sequence of increasing barriers, 
delimiting larger and larger subsets of state space. The simplest class of mathematically 
analyzable structures with this property are trees. Master equations on trees |T9|-p5| have 
propagators which decay algebraically in time, and a decay exponent (or exponents ||24|| ) with 



a linear or s-shaped temperature dependence ( depending on the particular model ). When 
simple 'magnetic' assumptions are introduced, one can reproduce the AC susceptibility 
curves [^J, the aging behavior ^-T2^], including changes of the apparent age due to a 
temperature step |3D| and age reset by temperature cycling [|3T|. As the predictions of 



hierarchical models are in good agreement with the experimental facts, a direct check of 
their basic assumptions seems a worthwhile endeavor. We note that as low temperature 
excitations in spin-glasses are small non-interacting clusters in a frozen environment fl6HT8 



parallel and sequential modes of relaxation must both be represented in the dynamics. This 
leads to the question of exactly what parts of state space, if any, hierarchical models describe, 
and to the allied, more general question, of the relation between state space phenomenologies 
and the widely used domain growth approaches. 

A numerical state space analysis is attempted here, by exhaustively enumerating and 
statistically analyzing millions to hundreds of millions of configurations close to deep energy 
minima. We do not try to study the state-space dynamics directly, since the problem 
sizes considered prevent us from calculating connectivity matrices and numerical solving the 
master equation for thermalization. Exhaustive search is insensitive to entropic barriers 



and gives therefore incomplete information with regard to dynamical properties. However, by 
including in the discussion previous studies of domain growth and by using simple heuristic 
arguments, some conclusions can be drawn about thermal (meta) stability. 

The extensive calculations required in this investigation were done on a Silicon Graphics 
Onyx parallel machine with 24 processors and 2 gigabytes of RAM at the University of 
Odense, Denmark, and on a Silicon Graphics 32 CPU Origin2000 with up to 32 GB of RAM 
kindly made available to this project by the SGI Advanced Technology Centre in Cortaillod, 
Switzerland. 

II. METHOD AND RESULTS 

Low temperature excitations in spin glasses are small connected domains of spins which 
are reversed relative to the their orientation in a very low energy configuration. This gives 
rise to a block structure in state space, where each block contains the dynamically available 
states of a single domain. Insight in the state-space geometry of one block can arguably be 
obtained by considering systems of size comparable to a domain: At low T this typically 
means up to 10 d spins, where d is the dimension. Even in such relatively small systems 
the number of configurations quickly becomes astronomical, precluding the possibility of 
examining all the configurations. Instead, one can use the 'lid method', which exhaustively 
visits relevant low-energy parts of the landscape. ]7|J|. A brief outline of the method is given 
here for completeness, while a more technical description, including details of the algorithm 
parallel implementation and performance statistics, is planned for a separate publication 

g. 

Consider a nearest neighbor walk which starts at a local energy minimum, or reference 
state, if), which, by definition, has zero energy. All states with energy above an energy 
'barrier', L are excluded from the walk. We let P^l be the set of all states which can be 
visited in such a walk. Following our previous notation |7| we refer to V^l as a pocket of 
depth L centered at if). As, in a thermal relaxation process, the 'typical' time needed to exit 
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*Pip,L (whether it be a first passage time or an average ) contains the factor exp(L/T), which 
is large at low temperatures, V is a bona fide candidate for a metastable region operationally 
defined by an energy barrier. 

The reference state ip is constructed by an iterative process. Initially, we pick a random 
state ipQ to start the exhaustive search. As soon as a new state with lower energy ipi is 
found, it becomes the current reference states and it is used as a starting point of a new 
search. At the n'th stage of the process ip n is the state of current lowest energy among 
all states examined. In this way the search algorithm 'digs' deeper and deeper into the 
landscape. The program stops when a certain fraction of the spins in the system (usually 
one half) is flipped without finding any state with energy lower than the current reference 
state. The latter is becomes then, by definition, the 'deep' minimum ip. Our exhaustive 



approach considerably differs from that of Vertechi and Virasoro |34|| , who sampled the 
distribution of energy barriers separating zero temperature solutions of the TAP equation: 
in our case the barrier is a parameter defining the pocket and all states in the pocket are 
found deterministically, allowing the calculation of the local density of states. 

The spin glass problem deals with a set of M Ising spins <jj = -1 placed on a 2D or 
3D cubic lattice with periodic boundary conditions. The energy of the x'th configuration, 
< x < 2^, is defined by the well known Hamiltonian [|35jl 

E(x) = -- Jijcri(x)aj(x), (2.1) 

where = Jji and where Jjj ^ only if spins i and j are adjacent on the grid. In this 
case, and for i < j, we take the J^-'s as independent gaussian variables, with zero average 
and variance J 2 = 1. By definition neighbor configurations are those which differ by the 
orientation of exactly one spin. 

Data are analyzed with an eye to the properties of mesoscopic state-space models. First 
and foremost we investigate, for E < L, the energy distribution of the states or local density 
of states T>(E, L) within a pocket of depth L. The integral of this quantity over its first 
argument is the state-space volume of the pocket, V(L). In most cases, the reference state ij; 



has the property that half of its spins can be overturned without finding any state of lower 
energy. Let L 1 / 2 be the corresponding energy barrier. For the smallest examples considered 
this barrier was found (within a small tolerance) to also allow the system to reach the spin 
reversed state — ip |36|. By chosing different initial condition for the search, while keeping 
the Jjj fixed, different deep reference minima can be found which are unrelated by global 
flips and which all have energies per spin at or below estimates of the ground state energy 
of spin-glasses |[37|| . We also present data for more shallow pockets, i.e. where the reference 
state is not particularly low-lying. We furthermore calculate the barrier values for which the 
current largest Hamming distance to the reference state increases its value, and the average 
number of neighbors to which a state is connected as a function of the lid. This number 
would be constant and equal to the number of spins if all neighbors were available, but as 
high lying states are discarded, the effective connectivity is much lower. 

All the investigations are performed for two and three dimensional lattices, which have 



similar patterns of low temperature relaxation pg| . In both dimensions we analyze, for a 
series of lattice sizes, 25 different realizations of the J^'s. As usual, disorder averages are 
performed. Furthermore we try to convey an idea of the system to system and pocket to 
pocket variation. In all cases we attempted to perform the exhaustive search up to the barrier 
value L\/2- Due to memory limitations, this was unfortunately not always possible for the 
largest among the systems considered: For the 10 2 systems we succeeded in 24 cases out 
of 25, missing by two spins in the unsuccessful case, while for the 6 3 systems we succeeded 
in 20 cases out of 25. The remaining instances had to be stopped at different stages of 
the calculation. The value of L\/2 and the size of the corresponding pockets greatly vary 
from sample to sample. For ease of display, we used the total volume of each pocket as a 
normalization factor for the corresponding local density of states. This creates vertical shifts 
of the data which are void of physical significance. 

Before considering any issue of interpretation, it is interesting to note that all the data 
can be parametrized rather simply: the local density of states is almost exponential, with a 
small systematic downward curvature (see Figs. 1-4). The available volume in the pocket is 
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also close to an exponential function of the lid energy, if one smooths out the jumps which 
occur when a new 'side pocket' appears as the barrier increases (see Figs. 7 and 8). Finally, 
the largest obtainable Hamming distance to the reference state grows exponentially with 
the barrier defining the pocket (see Figs. 5-6). All these properties are common to deep and 
less deep pockets. 

The exponential form of the local density of states is shown directly in Figs. 1 and 2 
for 10 2 and 5 3 systems respectively. In these plots the raw data are indicated by different 
symbols, one for each realization of the couplings, while the lines are calculated by fitting 
\og(T>(E, L max )) to a parabola in E. The coefficient of the second order term is typically 
20 times smaller than the coefficient of the linear term in 2D, and 50 times smaller in 3D. 
Therefore, the reciprocal of the linear coefficient basically describes the energy scale of the 
exponential density of state at low energies. These coefficients are plotted versus the system 
size in Figs. 3 and 4 for a number of 2D and 3D systems respectively. As discussed later, 
they provide an estimate of the kinetic transition temperature TJ. at which the system looses 
its thermal (meta) stability P~8"fl . 

Figs. 5 and 6 describe 2D and 3D systems respectively. The ordinate is the Hamming 
distance to the reference state, (scaled for convenience to the unit interval), while the abscissa 
is the average energy barrier which must be overcome to achieve that Hamming distance. 
Each data set represents a different system size, and in each case the average is performed 
over 25 realization of the disorder. 

In 3D we additionally performed two different types of investigations: 1) for one partic- 
ular realization of the 5 3 systems, we varied the initial conditions of the search leading to 
the reference state and 2) for one particular realization of a 6 3 system we studied the local 
density of states for 'shallow' minima. As a few percent of the states are energy minima 
it is not surprising that different quenches lead to different final states. It is however 
interesting that careful optimization similarly leads to different reference states, which are 
unrelated by symmetry and separated by barriers > Li/ 2 - This is shown in Fig. 7, where 
the volume V of three different deep pockets (for the same Jy) is plotted vs. L. To avoid 
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partly overlapping data points we multiplied the data by different factors, as indicated in 
the figure itself. 

In the second investigation we consider a sequence of five shallow pockets: The first data 
set is generated by starting the search in a high lying minimum, and stopping it as soon 
as a lower minimum is found. This newly found minimum then serves as a starting point 
for the second stage of the search, leading to the states in pocket #2, and so on. Figure 8 
shows the local density of state found in each of those pockets. The local density of states 
as well as the volume (not shown) are still close to exponential functions of their arguments. 
Indeed, this behavior appears in all the pockets we examined, regardless of the energy of 
the reference state. 



III. DISCUSSION 



Most theoretical descriptions of spin glasses either use insights and numerical observa- 
tions on the morphology of excitations at low temperature p^ |TT| , [T6|] or build on ideas about 
the structure of state-space PJ2^- p8| , |30| , pl] , |39| . The first method is rooted in a renormal- 
ization group approach developed by Mc Millan f40| and Bray and Moore [41 1 . It includes 
droplet PlJlOll and domain |lTJ theories, which emphasize correlation functions and their 



temperature and field dependence. The early work of Dasgupta et al. |16[] is mainly numer- 
ical, but already contains interesting theoretical observations on the excitation morphology. 
Other approaches are either inspired || by the equilibrium solutions of the mean field model 



42[ or use a purely dynamically motivated lumping procedure to produce hierarchical models 

Campbell's 



of state-space, within the general approach of broken ergodicity | 

approach |39|] describes the spin-glass transition as a percolation transition in state space. 
The hierarchical scenario described by by Lederman et al. || and Vincent et al. shares 
with the models of refs. p6|- |2^j30|j31[] , the idea that, as the temperature is lowered, more and 
more details of state space emerge. The difference is 1) in the interpretation of the 'vertical' 
direction in the tree, which is a magnetic overlap in one approach and an energy in the 
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other, and 2) in the importance attached to the local density of states. In the approach of 
Joh et al. p5| , an ultrametric tree describes a manifold of states of constant magnetization, 
and the response behavior to a field step —>■ H is associated to a transfer of probability 
from the M = manifold to a manifold of states having the field cooled magnetization 
Mpc- A phenomenological treatment of relaxation in spin glasses introduced by Bouchaud 



44fl builds on the idea that the distribution of barrier heights within a trap is exponential. 
Even though the approach differs from that of models where the barrier parametrizes state 
space, an underlying similarity is that, in any tree structure with a minimum of regularity, 
the distribution of the barriers delimiting subtrees is indeed typically exponential. 

Domain theories and hierarchical models have mainly been regarded as competing ap- 



proaches [43,45,46], and the discussion over their relative merits has been intertwined with a 



a continuing debate on the nature of equilibrium states of spin-glasses |14 |. Even though the 
latter question has great theoretical interest, low temperature relaxation never probes equi- 
librium properties, as for example indicated by the fact that similar physical behaviors (e.g 
power law relaxation and aging) characterizes system of different dimensionality. We would 
argue that much of the low temperature phenomenology can be understood by looking at 
the physics on small length scales, that the dichotomy between real space and configuration 
space descriptions is moot and that (some) hierarchical approaches are justified regardless 
of the relevance of mean field behavior for short range spin-glasses. 

Under the assumption that the set of spin configurations available to a domain is similar 
to the configuration space of a small systems, the growth of the average domain size V rs (t) 
should match the growth of the configuration space volume V cs (t), which can be visited by 
a Metropolis algorithm in t time steps. At low temperatures, the former quantity grows 
algebraically in time and can be written as V rs (t) cx exp(6(t,T)), where b is the barrier 



b(t,T) « Tln(i) which thermal fluctuations typically overcome in time t ||18|| . We have 
unpublished evidence that V cs grows algebraically as well. The same conclusion can be 
reached from the present evidence, if one makes the ( reasonable ) additional assumption 
that, starting from zero energy, states of energy b are visited on a time scale a(T)\nt. 
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(An Arrhenius behavior corresponds to ot(T) oc T, and we expect ot(T) to be close to 
a linearly growing function of T). Since, as our present data show, the volume of the 
configuration space 'below' b increases exponentially with b, we recover V cs (t) « t Ca<yT \ 
where C is some constant. Associating the configuration space volume with the spins in 
a cluster thus implies that V cs oc Vjr s for some constant (3, which again implies a strongly 
constrained and predominatly sequential dynamics within a cluster. We recall that several 
very low energy configurations unrelated by flip symmetry exist, as clearly shown by Fig. 
7. This is somewhat reminescent of mean-field behavior, and consistent with the results of 



numerical investigations in small systems |47j showing a non-trivial Parisi overlap function. 



It could also be related to the chaotic sensitivity of thermal correlations || in spin glasses: 
Since the barriers among multiple deep minima are high, thermal fluctuations are confined 
to the surroundings of one specific state. However, field or temperature changes which 
destroy local equilibrium and change the barrier structure can induce transitions from one 
deep pocket to another, leading, in real space, to a partial obliteration of the pre-existing 
thermalized domain structure and thus to thermal chaos. 

Let us now consider how our present numerical evidence specifically relates to hierarchi- 
cal models. It must be born in mind that these models are intended as a coarse grained 
description of state space: some of their features i.e. the lack of translational invariance, the 
form of the local density of states, and the relation between the barrier and the Hamming 
distance to the starting configuration, are of central importance to the relaxation behav- 
ior. By way of contrast, the lack of loops characterizing tree graphs only approximates the 
sparsity of the connection matrix and cannot be an exact property of the state space of mi- 
croscopic models. A tree obtained by successive branchings from a 'top' node can be viewed 
as 1) a hierarchy of energy or free energy barriers, with all the states lying at the lowest 
level p!9|j20|] , or 2) all nodes in the tree may represent lumped physical states |2T|-f24|j. In the 



first case (and not in the second) the system has an ultrametric distance: for two arbitrary 
bottom nodes this distance is just the number of levels up to the top of the smallest subtree 
containing both nodes. In mean-field models one identifies the index of the hierarchy with 
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a magnetic overlap, while in the approach developed in Refs. p6|- p8| , |30| , pT| , the index of the 
hierarchy is an energy barrier ||48|| , and the statistical weight attached the lumped nodes 
increases exponentially with this barrier. This last feature agrees with the fact that both 
the local density of states (Figs. 1 and 2) and the state-space volume of pocket (Fig. 7) grow 
exponentially with their respective arguments. The magnetic properties of the models hinge 
on the disorder averaged overlap with the reference state, which was assumed for simplicity 
to increase linearly along the 'bottom' nodes of the tree The assumption implies that 



the largest achievable Hamming distance within a pocket should grow exponentially with the 
confining barrier. As shown in Figs. 5 and 6, such an exponential relation is confirmed by 
the present investigation. Let us finally consider the issue of thermal met ast ability, which, 



as first noticed in refs. pT|j22| , characterizes models with an an exponential density of states 
oc exp(E/Tk). At the 'glass transition temperature' T = the equilibrium probability 
becomes strongly biased towards high lying, rather than low lying configurations, basically 
due to a change of sign in the exponent of P(E) oc exp(—E/T + E/T^). If, as we here 
suggest, these models describe the physics of a trap, this trap must cease to be a thermal at- 
tractor when the temperature exceeds T^. Clear concurring evidence is the dramatic change 



in domain growth from slow to fast, taking place at a well defined temperature - which 
in 3D is close to 0.8 J ( and close to the actual critical temperature ), while in 2D it is close 
to 0.6J. The change is not simply related to the existence or absence of a true equilibrium 
transition, since it appears already at short times and independently of dimensionality. Fig- 
ures 3 and 4 display, as a function of the linear size of the lattice, the 'kinetic transition 
temperatures' naively calculated as the reciprocal of the scale of the exponentially growing 
density of states, for a number of realizations of spin-glass systems in two and three spatial 
dimensions. If the confining barriers of the system were purely energetic, the density of 
states would contain all relevant information. The TVs would then be independent of the 
system size and coincide with the kinetic transition temperatures. In reality, while the range 
of the estimated Tj. includes the correct values, the data have a clear downward trend, lead- 
ing to the expected conclusion that entropic confinement becomes important as the linear 
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size of the system grows beyond a certain limit, in this case ^4 — 5. What likely happens 
in a large system is that, when T c is approached from below, small clusters melt and start 
to interact strongly with one another other via the large amount of loose spins generated in 
the process. According to Fig. 4 this would happen, for 4 3 systems, at T ~ 0.8. 

In conclusion, we have argued that 1) the state space configuration corresponding to 
a small isolated domain can be adequately described by a hierarchically organized set of 
states, and 2) that the loss of metastability of these domains is due to their exponentially 
growing density of states which plays an important, albeit not exclusive, role for the thermal 
metastability of system as a whole. |49]. Similar results appear to hold for other complex 
systems with multiple local energy minima, as shown by the previous analysis of the TSP 
problem J7j and by ongoing investigations of covalent network models of glasses [f)2] 
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FIGURES 

10x10 spin glass 




energy (J) 



FIG. 1. The local density of states within a pocket centered at a deep energy minimum is 
shown as a function of the energy in a semilogarithmic plot, for 25 different realizations of the 
couplings in a 10 x 10 system. The dotted lines are fits of logT> to a parabola. As we have 
normalized the integral of the density of states to one, the differences in magnitude among the 
data sets directly reflect the size of the corresponding pockets. 



17 



10° 



5x5x5 spin glass 



~o 10-4 r 



CO 
C 
CD 



03 
o 



O 



CO 



■1— • 

■1— • 
CO 



10 



,-2 





10" 6 



!>4 ij)v' c ■ i» l 1 1 1 1 1— 

2 4 6 8 10 12 



14 



energy (J) 



FIG. 2. The local density of states in a pocket centered at a deep energy minimum is shown 
as a function of the energy in a semilogarithmic plot, for 25 different realizations of the couplings 
in a 5 x 5 x 5 system. The dotted lines describe fits of log T> to a parabola. As the integral of the 
density of states is normalized to one, the differences in magnitude among the data sets reflect the 
size of the pockets. 
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FIG. 3. Each dot marks the estimated kinetic transition temperature of one realization of 
a 2D system of a given linear size. The average value of is indicated by a circle. The T^'s are 
calculated by expanding logP to second order in E and taking the reciprocal of the coefficient of 
the (dominating) linear term. 
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FIG. 4. Each dot marks the estimated kinetic transition temperature of one realization of 
a 3D system of a given linear size. The average value of is indicated by a circle. The T^'s are 
calculated by expanding logP to second order in E and taking the reciprocal of the coefficient of 
the (dominating) linear term. 
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FIG. 5. The ordinate is the largest Hamming distance to a reference state of very low energy 
which can be achieved without overcoming the energy barrier given by the abscissa. The data are 
averaged over 25 different disorder realizations of a 2D system. Note that after a short transient, 
the dependence is simply exponential. 
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FIG. 6. The ordinate is the largest Hamming distance to a reference state of very low energy 
which can be achieved without overcoming the energy barrier given by the abscissa. The data are 
averaged over 25 realizations of a 3D system. 
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FIG. 7. The plot shows the characteristic exponential dependence of the accessible number 
of states on the energy barrier for three disjoint pockets belonging to the same realization of the 
couplings in a 5 3 system. For convenience, the data have been multiplied by different factors as 
indicated in the figure itself. 
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FIG. 8. The plot describes the local density of states in a series of 'shallow' pockets belonging 
to a single 6 3 system. The rightmost data set is obtained by starting the exhaustive search in 
a rather poor local energy minimum (energy = — 1. 605716 J/spin ) and stopping it as soon as a 
lower energy minimum is found. This new minimum (energy = — 1.610523 J/spin ) is the starting 
point for the search producing the second set of data, which again stops when a better minimum 
is found. All shown data sets are obtained in a similar fashion. The values of the energy per 
spin of the reference states - besides the first two just mentioned - are —1.632942, —1.637671 
and —1.643615 J/spin. For convenience the energy of the very lowest energy configuration is used 
as the origin of the abscissa. The densities are normalized by the total number of states in the 
corresponding pocket. 
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